Conventionally, a technique for analyzing an image of a known marker captured by a camera, estimating a position and a posture of the marker on the image, and determining a relative position and the relative posture between the marker and the camera has been developed.
However, when the marker on a video is applied to another image, unless the image is combined to perfectly fit the position and the posture of the marker by accurately estimating the position and the posture, the applied image can be seen as if it is swinging or distorted. Therefore, in order to improve the estimation accuracy of the position and the posture of the image, a known pattern image is mostly used as the marker. However, when the recognition of the pattern image is performed, since an estimation limit is generated in exchange for improving the estimation accuracy, a sense of incongruity in a video as a combining result is caused.
For example, it will be described that a combined video is created by pasting another image on the marker held with a performer's hand. When the performer gradually raises the marker being directed perpendicularly to the camera in order to be captured by the camera, the pattern of the marker comes into sight gradually stretching from the collapsed state. In this case, unless the pattern is seen to some extent, a pattern recognition process cannot be performed, but depending on a recognition method. Therefore, although the marker appears, the pattern recognition cannot be performed and it is determined that the marker does not exist.
Consequently, on the combined image, as the performer raises the marker, the pattern of the marker comes into sight, whereas the video is created on which the marker is suddenly replaced by another image when the pattern recognition process can be performed. In this situation, not only the sense of incongruity is caused by discontinuity of a board (the marker) held with the performer's hand, but the pattern image, which is used for the marker and irrelevant to the combined video, appears.
The problem that the marker is seen is similarly caused in a mixed reality device showing a video combining a reality space and a virtual image to a player wearing a head mound display.
For example, Patent Document 1 discloses a technique for hiding a marker on a combined image by preparing beforehand a reality space image on which the marker is not placed and using the reality space image having no marker for the reality space which has been combined on a virtual image by placing the marker. However, the technique disclosed in Patent Document 1 cannot stably hide the marker, since it is assumed that the marker is continuously detected, when the marker is difficult to detect due to the change of the angle at which the marker can be seen. The change of the angle is caused when the player bends down.
Furthermore, Patent Document 2 discloses a technique for not spoiling a total design, such as an advertising design, with an uninteresting mark by changing a color of a marker itself. However, since the combined image is created after the marker is recognized, until the recognition is completed, the marker is simply output as an output image on the combined image.